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DETAILED ACTION 



Continued Examination Under 37 CFR LI 14 

1 . A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.1 14, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.114. Applicant's submission filed on 03/18/05 has been entered. 



Claim Rejections - 35 USC § 102 
2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

(e) the invention was described in (I) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351 (a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 



The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) and the Intellectual Property and High Technology Technical Amendments Act of 2002 
do not apply when the reference is a U.S. patent resulting directly or indirectly from an 
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international application filed before November 29, 2000. Therefore, the prior art date of the 
reference is determined under 35 U.S.C. 102(e) prior to the amendment by the AIPA (pre-AIPA 
35U.S.C. 102(e)). 

3. Claims 1-6, 8-1 1 and 16-19, 22-24 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Schofield et al. (US 6.498,620 B2). 

Re claim 1, Schofield discloses an image processing apparatus comprising: an image 
processing part (18 of fig. 5) for receiving images captured by a plurality of cameras (14 and 16 
of fig. 7) shooting surroundings of a vehicle (12 of fig. 1) to generate a synthetic image (col. 5, 
lines 59-64) viewed from a virtual point of view (col. 6, lines 1-4) from these camera images 
(col. 5, line 65-col. 6, line 4), wherein the image processing part (18 of fig. 5) changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
of view in accordance with a running state of the vehicle ( col. 21, line 41-col. 22, line 6; note 
the image processor (18) responds to the temporal and spatial patterns of infrared signals 
detected by image capture devices (cameras 14, 16) in order to determine (selected) the speed 
and distance and, thereby, the separation of the vehicles as well as the rate of change of 
separation of the vehicles as considered a running state of the vehicle). 

Moreover, Schofield further discloses the image processing part (18 of fig. 22) changes at 
least one selected from a position (col. 21, lines 25-41, e.g. the image processing changes a 
position in accordance with a running state of the vehicle), a direction of a line of sight (col. 21, 
line 67-col. 22, line 6,. e.g. changing a field of view from rearward to forward of, or to the side 
of, the vehicle; see also col. 14, lines 1-29), and a focal length of the virtual point of view in 
accordance with a running state of the vehicle (cameras 14 and 16 of fig. 24, e.g. the cameras 14 
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and 16 inherently have a focal length and virtual point of view, so when the vehicle changes 
from one location to other location the focal length of the cameras also change); controlling 
capturing of an image outside a view range (cameras 14 and 16 of fig. 4, e.g. the cameras 14 and 
16 captures the images outside a view range of the driver when the vehicle moving forward 
direction as consider running state, and the captured images can be displayed on the display 20' 
of fig. 10) of the virtual point of view in accordance with a running state of the vehicle; the 
image processor (18 of fig. 5) can generate an image including a first image viewed from the 
virtual point of view (14 Left of fig. 5) and a second image viewed from a different viewpoint 
and a different model (14 Right of fig. 5., see also 14A, 19 of fig., e.g. the camera 14A Left 
captures the fist image at different angle from the camera 14A Right that captures the second 
image, the cameras 14A Right and Left are have different viewpoint from each other). 

Re claim 2, Schofield further discloses wherein the image processing part (18 of fig. 5) 
changes at least one selected from a position, a direction of a line of sight, and a focal length of 
the virtual point of view in accordance with a running speed of the vehicle (fig. 10 and fig. 11, 
note when the vehicle turns to the left or right, a position is changed so that the image capture 
devices (14 and 16 of fig. 5) picks up a new images based upon the changed position). 

Re claim 3, Schofield further discloses wherein the image processing part (18 of fig. 5) 
changes at least one selected from a position, a direction of a line of sight, and a focal length of 
the virtual point of view, and controls capturing of an image outside a view range of the changed 
virtual point of view (col. 14, lines 1-29, see also fig. 7, note distances R, S, PI and P2). 
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Re claim 4, Schofield further discloses wherein the image processing part (18 of fig. 5) 
controls the capturing of an image outside a view range of the changed virtual point of view by 
changing a model for image synthesis (an object). 

Re claim 5, Schofield further discloses wherein the image processing part changes at least 
one selected from a position, a direction of a line of sight, and a focal length of the virtual point 
of view in accordance with a steering angle of the vehicle (XVII of fig. 12, see also col. 17, lines 
31-36). 

Re claim 6, Schofield further discloses wherein the vehicle includes an object detection 
sensor (176 of fig. 21, note the object sensor 176 may be a distance-measuring device, such scan 
active infrared sensor, an ultrasonic sensor, a radar sensor, or the like. Such object sensor is 
especially useful in determining the separation distance between the vehicle and objects in front 
of the vehicle. Preferably, object sensor 176 has a sensing field of view that is substantially 
coincident with the field of view of one or more of the image capture devices 14, 16) for 
detecting an obstacle, and the image processing part changes at least one selected from a 
position, a direction of a line of sight, and a focal length of the virtual point of view in 
accordance with results of detection by the object detecting sensor (176 of fig. 21). 

Re claim 8, Schofield further discloses an image processing apparatus comprising fig. 21) 
an image processing (18 of fig. 21) part for receiving images captured by a plurality of cameras 
(14 and 16 of fig. 21) shooting surroundings of a vehicle (10 of fig. 1) to generate a synthetic 
image viewed from a virtual point of view from these camera images, wherein the image 
processing part (18 of fig. 21) controls capturing of an image outside a view range of the virtual 
point of view in accordance with a running state of the vehicle (col. 14, lines 1-7). 
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Re claim 9, Schofield further discloses a monitoring system comprising: a plurality of 
cameras (14 and 16 of fig. 21) shooting surroundings of a vehicle; an image processing part 
(18of fig. 21) for receiving images captured by the plurality of cameras to generate a synthetic 
image viewed form a virtual point of view from these camera images, a display part (20 of fig. 7) 
for displaying the synthetic image, wherein the image processing part changes at least one 
selected from a position, a direction of a line of sight, and a focal length of the virtual point of 
view in accordance with a running state of the vehicle (100 and 174 of fig. 21). 

Re claims 10 and 16, Schofield further discloses an image processing apparatus 
comprising an image processing part (18 of fig. 21) for receiving images captured by a plurality 
of cameras (14 and 16 of fig. 2 1) shooting surroundings of a vehicle to generate a synthetic 
image from these camera images, wherein the image processing part (18 of fig. 21) generates an 
image including a first image (the captured image is from the camera 14 or 16) as the synthetic 
image, the second image being viewed fig. 3, the right image being viewed on the display is 
different from the left image) from a viewpoint that is different from the virtual point of view of 
the first image in at least one selected from a position, a direction of a line of sight and a focal 
length, or the second image being different from the first image in a model. 

Re claim 11, Schofield further discloses wherein the second image is at least one of the 
camera images (fig. 3). 

Re claims 17 and 23, Schofield further discloses an image processing apparatus 
comprising: an image processing part (18 of fig. 2) for receiving images captured by a plurality 
of cameras (14 and 16 of fig. 7) shooting surroundings of a vehicle (10 of fig. 1) to generate a 
synthetic image from these camera images, wherein in the synthetic image, the image processing 
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part (20 of fig. 5 and a display 20 of fig. 3) displays at least a part of a vehicle region (42 of fig. 
3) where the vehicle is present, and an attention drawing region (50 and 52 of fig. 3) for drawing 
attention in which at least a part of the surroundings of the vehicle is shown, wherein the 
attention drawing region includes at least a part of a blind spot region around the vehicle that is 
not shot by any of the cameras (20 of fig. 3, note the display (20) is displaying the image that 
includes a part of blind spot, a part of a blind spot regions around the vehicle that is not shot by 
any of the cameras as considered harsh marks 70a and 70b of fig. 6, harsh marks are displayed 
on the display 20" to indicate the blind spot drawing regions that not shot by any cameras, col. 
17, lines 10-64). 

Re claim 18, Schofield discloses wherein the synthetic image is an image viewed from a 
virtual point of view that is set above the vehicle (fig. 3). 

Re claim 19, Schofield further discloses wherein the image processing part displays an 
illustration image or an actual image of the vehicle on the vehicle region (20 of fig. 3, see also 
col. 7, line 47-col. 8, line 12). 

Re claim 22, Schofield further discloses wherein the image processing part determines a 
range of a region obtained by combining the blind spot region and the vehicle region, using 
region data showing a projection region of the vehicle in each camera image (76 of fig. 14., see 
also col. 9, lines 7-1 1). 

Re claim 24, Schofield discloses an image processing apparatus comprising an image 
processing part (18 of fig. 5) for receiving images captured by a plurality of cameras (14 and 
16of fig. 5) shooting surroundings of a vehicle to generate a synthetic image from these camera 
images, wherein the image processing part (18 of fig. 5) generates the synthetic image, using a 
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mapping table (col. 14, lines 30-56) including first mapping data describing a correspondence 
relationship between pixels of the synthetic image and pixels of the camera images (Left image 
and Right image are from the outputs of the cameras 14 in figure 5, see also fig. 3), and second 
mapping data describing an identifier showing that a pixel of the synthetic image corresponds to 
pixel data other than the camera images (58 of fig. 3), and Schofield further discloses wherein 
the pixel data other than the camera images show the vehicle or a blind spot region that is present 
in at least a part of the surroundings of the vehicle (44, 46, 48 of fig. 3, e.g. the blind spot region 
is between 50 and 52 of fig. 3 and the surrounding cars are also displayed). 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 10-15 and 26-27 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schofield et al. (US 6,498,620 B2) in view of Shimizu (US 5,796,991). 

Re claims 10-15, and 26-27, Schofield teaches an image processing apparatus comprising 
an image processing part (18 of fig. 5) for receiving images captured by a plurality of cameras 
(14 and 16 of fig. 5) shooting surroundings of a vehicle to generate a synthetic image from these 
camera images, wherein the image processing part (18 of fig. 5) generates the synthetic image, 
using a mapping table (col. 14, lines 30-56) including first mapping data describing a 
correspondence relationship between pixels of the synthetic image and pixels of the camera 
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images (Left image and Right image are from the outputs of the cameras 14 in figure 5, see also 
fig. 3), and second mapping data describing an identifier showing that a pixel of the synthetic 
image corresponds to pixel data other than the camera images (58 of fig. 3); wherein the pixel 
data other than the camera images show the vehicle or a blind spot region that is present in at 
least a part of the surroundings of the vehicle (70A and 708 of fig. 6). 

It is noted that Schofield does not teach the image processing part stores a predetermined 
image other than the camera images, and with respect to the pixel of the synthetic image, the 
second mapping data describe coordinate values corresponding to the pixel in the stored 
predetermined image; wherein the first image is a close view image showing the vehicle and 
surroundings thereof, and the second image is a distant view image showing an area distant from 
the surrounding area of the vehicle that is shown by the close view image; wherein the image 
processing part arranges the distant view image around the close view image in the synthetic 
image; wherein the distant view image is an image having continuity with the close view image 
as claimed However, Shimizu further discloses wherein the image processing part stores a 
predetermined image other than the camera images (CG MODEL DATABASE, 244 of fig. 7), 
and with respect to the pixel of the synthetic image (213 of fig. 7), the second mapping data 
describe coordinate values corresponding to the pixel in the stored predetermined image (CG 
NAGE FORMING UNIT, 235 of fig. 7); wherein the second image is at least one of the camera 
images (201 L and 20 1R of fig. 7); wherein the first image is a close view image showing the 
vehicle and surroundings thereof, and the second image is a distant view image showing an area 
distant from the surrounding area of the vehicle that is shown by the close view image tfig. 6 A, 
fig. 10A-10E); wherein the image processing part arranges the distant view image around the 
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close view image in the synthetic image (figs. 6A and 6B); wherein the distant view image is an 
image having continuity with the close view image (figs. 10A-10E); wherein the first image 
shows at least a part of the vehicle and at least a part of the surroundings of the vehicle, and the 
second image is obtained by enlarging at least a part of the region shown by the first image (figs. 
10A-10E). Taking the combined teachings of Schofield and Shimizu as a whole, it would have 
been obvious to one of ordinary skill in the art to incorporate the teachings of Shimizu into the 
image processing part (18 of fig. 5) of Schofield for the same purpose of synthesizing the images 
with the predetermined model. Doing so would provide a natural appearance among images of 
virtual objects and an improved simulation environment. 

6. Claim 28 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schofield et al. 
(US 6,498,620 B2) in view of Shum et al. (US 6,271,847). 

Re claim 28, Schofield discloses an image processing apparatus comprising an image 
processing part (18 of fig. 5) for receiving images captured by a plurality of cameras (14 and 16 
of fig. 5) shooting surroundings of a vehicle to generate a synthetic image from these camera 
images, wherein the image processing part uses mapping data (col. 14, lines 30-56 ) but not 
describing a correspondence relationship between pixels of the synthetic image and a plurality of 
pixel data including one or both of pixel data of the camera images and pixel data other than the 
camera images, and describing a rate of necessity with respect to each of the pixel data, and 
weights each pixel data in accordance with the rate of necessity, thereby generating the pixel data 
of the pixels of the synthetic image. 
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However, Shum teaches describing a correspondence relationship between pixels of the 
synthetic image and a plurality of pixel data including one or both of pixel data of the camera 
images and pixel data other than the camera images, and describing a rate of necessity with 
respect to each of the pixel data, and weights each pixel data in accordance with the rate of 
necessity, thereby generating the pixel data of the pixels of the synthetic image (fig. 1 1 and 12). 
Therefore, taking the teachings of Schofield and Shum as a whole, it would have been obvious to 
one skill in the art to incorporate the teachings of Shum into Schofield to perform the process of 
mapping weighted pixels and rate. Doing so would provide for better blending of images having 
radically different scales (i.e. zoom). 

7. Claims 1, 7 and 29 rejected under 35 U.S.C. 103(a) as being unpatentable over Daily et 
al. US 6,317,127) in view of Schofield et al. (US 6,498,620 B2). 

Re claims 1, 7 and 29, Daily teaches an image processing apparatus (fig. 1) comprising 
an image processing part (48 of fig. 1) for receiving images captured by a plurality of cameras 
(12 of fig. 1) shooting surroundings of airplane to generate a synthetic image (50 of fig. 1) from 
these camera images, wherein the image processing part includes an original mapping table (80 
of fig. 5), cuts out a mapping table describing a correspondence relationship between pixels of 
the synthetic image and pixels of the camera images (figs. 6 and 7), and generates the synthetic 
image, using the cut-out mapping table (108 of fig. 7); wherein the image processing part (40 of 
fig. 1) includes an original mapping table (fig. 5) and generates a synthetic image using a 
mapping table (108 of fig. 7) that is cut out from the original mapping table, and the image 
processing part changes at least one selected from a position, a direction of a line of sight, and a 
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focal length of the virtual point of view by changing the mapping table to be cut out from the 
original mapping table (106, 1 14 and 1 16 of fig. 7). 

It is noted that Daily does not particularly teaches the cameras are mounted on or 
attached to the vehicle to capture the surroundings as claimed. 

However, Schofield teaches cameras (14 and 15 of fig. 1) are mounted on the vehicle and 
capturing images surroundings the vehicle. Therefore, taking the combined teachings of Daily 
and Schofield as a whole, it would have been obvious to one of ordinary skill in the art to 
incorporate the teachings of Daily and Schofield as a whole, it would have been obvious to one 
skill in the art to incorporate the teachings of Schofield into the image processing apparatus of 
Daily to generate the synthesized image from virtual point of view from the cameras. Doing so 
would improves performance by further reducing blind spots at the side and rear of the vehicle. 

8. Claims 30 and 3 1 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Schofield et al. (US 6,498,620 B2) in view of Ijima et al. (US 5,973,726). 

Re claims 30 and 31, Schofield teaches the synthesized image above but not the virtual 
point of view has been viewpoint-converted from an actual viewpoint as claimed. 

However, Iijima teaches the virtual point of view has been viewpoint-converted from an 
actual viewpoint (38 of fig. 1, e.g. the corresponding point extracting portion is extracting 
(converting) the virtual point of view from the actual viewpoint of the cameras (10L and 10R of 
fig. 1) and then forming the plane image from the virtual point of view of the images from the 
cameras; see also col. 9, lines 15-21). 



Application/Control Number: 09/807,654 Page 13 

Art Unit: 2613 

Therefore, taking teachings of Scho field and Ijima as a whole, it would have been 
obvious to one of ordinary skill in the art to incorporate the teaching of Ijima into the image 
processing apparatus of Schofielf for generating the synthesized image from the virtual point of 
view from the cameras. Doing so would provide the apparatus to generate the synthesized image 
for display without losing continuity of images and degrading the quality of image. 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

Dearth (US 3,750,166) discloses a pilot data system. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tung Vo whose telephone number is 571-272-7340. The 
examiner can normally be reached on Monday-Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Chris. Kelley can be reached on 571-272-7331. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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